1. Field of the Invention
The present invention relates to an improvement of a neural network system.
2. Description of the Related Art
A so-called "neural network system" is known which is analogous to the neural networks of animals. Neural network data-processing systems are disclosed in Kazuyuki Aihara, Neural Network Computer, Tokyo Denki University Press, 1988, and in some other publications.
FIG. 1 schematically shows a circuit equivalent to an analog neural network. The amplifiers shown in FIG. 1 correspond to the neurons (i.e., neural cells) of the neural network of an animal. The feedback line of the line, the resistor wij, and the input line correspond to an excitatory synaptic connection in the animal neural network. To constitute a connection equivalent to a inhibitory synaptic connection, the inverted output voltage -xj of the j-th amplifier is applied to the input of the i-th amplifier.
The current .SIGMA..omega.ij.multidot.xj flowing through the resistor wij is added to the current Ii supplied to the i-th amplifier from an external device. The sum of the currents .SIGMA..omega.jxj and Ii flows to the ground through a capacitor Ci and a resistor .rho.i. An input voltage ui is applied to the i-th amplifier.
In most cases, the output function of the i-th amplifier is a sigmoid function represented by the solid line or the broken line in FIG. 2. The i-th amplifier outputs a voltage xi which corresponds to the input voltage ui.
The operating characteristic of the neural network shown in FIG. 1 can be represented by the following equations: ##EQU1##
If the synaptic connection in the neural network is a symmetrical one (wij=wji, wii=0), the neural network has the energy function (Liapunov function) defined by the following equation: ##EQU2## The neural network operates to reduce an energy value E. The local minimum value of energy E corresponds to an attracter (a steady state) of the circuit.
When the circuit illustrated in FIG. 1 is used to process information, it is necessary to determine an energy function which indicates an optimal state of the circuit when it takes the minimum value. The energy function can be determined since the minimum value of the energy E corresponds to an attracter of the circuit. The energy function, thus determined, is changed to the form of Equation (4), whereby the synaptic load wij and the input current Ii are set to the circuit shown in FIG. 1. Then, the circuit is operated, whereby the solution of Equation (4) is obtained from the output voltages of the amplifiers.
Most neural networks each have a plurality of attracters. Hence, the energy function represented by Equation (4) has valleys E1 and E2 as is illustrated in FIG. 3. It is desirable that the energy of the network 10 is transferred along route A to the deepest valley E1 as fast as possible, thereby to bring the circuit into a steady state. However, there is the possibility that the energy of the neural network is transferred along route B or C, depending upon, for example, the initial value set to the i-th amplifiers or the value of the output functions of the amplifiers. If the energy is trapped in the shallow valley E2, the neural network is likely to output an inappropriate solution.
Two methods can be employed to prevent the network from outputting an inappropriate solution. The first method is to change either the initial values of the amplifiers or the output function. The second method is to input noises to the amplifiers, so as to prevent the energy of the network from being trapped in the shallow valley E2. Either method, however, is time-consuming and prevents the neural network from processing information at high speed. The use of either method is fatal to the neural network system employed as an on-line, real-time data processing system.
There are systems which are designed to monitor and control distribution systems such as the combustion temperature in a gas turbine, the temperature in a boiler evaporation tube, the temperature in a fuel-cell stack, the temperature in a furnace, the temperature in a chemical reaction tank, the temperature in a room, and the pressure in a water-supply network.
Generally, such systems receive a number of analog input signals from, for example, sensors and employ, as control values, the maximum signal value, the intermediate signal value, the minimum signal value, the mean value of all analog input signals, except the maximum input signal and the minimum input signal, and/or the average value of several analog input signals having values similar to the intermediate value. In such systems, it is necessary to determine signals at the highest level, intermediate level, lowest level and the like from the input analog signals. Conventionally, such determination is performed by comparing the signal levels of the analog input signals repeatedly. Therefore, in the conventional method, it takes a long time to select the signal at a desired level from the plurality of the input analog signals. The Traveling-Salesman Problem to find the shortest possible route along which a salesman can visit all cities he must visit, and the Lot-Production Problem to determine the best possible sequence of producing lots of different articles in a single assembly in order to save time, are called the optimization problems. The optimization problem is solved by computing the evaluation index for any possible combination of items. Hence, the more items there are, the more time the conventional data-processing system will require to solve the optimization problem. More precisely, when the items increase N times in numbers, for example, the time the system requires to solve the problem will increase about N.sup.3 times. Therefore, it is practically impossible for an ultra-high-speed computer to find a solution to the optimization problem when there are tens of items involved.
The uses of neural networks to obtain such kind of solutions are disclosed in Hopfield and Tank, "Neural" Computation of Decisions in Optimization Problem, Biol. Cyvbern. Vol. 52, 1985, pp. 141-152.
However, no neural networks have been developed which can determine the signal at a desired signal level.
The neural networks, available at present, are not designed to find solutions which result in an evaluation index optimal for various conditions such as the deadline dates of visits to cities and the deadlines for delivering products.